Manipulation of objects can be a function of a wide class of robotic systems (e.g., manipulation systems). For instance, a typical closed-loop manipulation system can include at least one optical sensor (e.g., depth sensor) that perceives and interprets real-world scenes and a physical manipulator (e.g., actuator, robotic arm, etc.) that can reach into the scenes and effect change (e.g., pick up and move a physical object). An ability to perform useful tasks with a physical object can depend on the ability of the robotic arm to manipulate the object with sufficient accuracy for the task. The accuracy can depend on a mapping between a coordinate system of the depth sensor and a coordinate system of the robotic arm. For instance, a mapping can be created between coordinates extracted from a depth image generated by a depth sensor and Cartesian positions of a robotic arm in a workspace. Such mapping between coordinate systems can also be referred to as registration. Precise and accurate mapping between depth sensors and robotic arms is a long-standing challenge in robotics and is relevant for substantially any robotic system that has one or more depth sensors. Errors in the mapping can result in inaccuracies of the overall manipulation system.
Several independent sources of registration errors can each contribute to overall system inaccuracy. In a typical naïve setup, a Cartesian coordinate system oftentimes is chosen to represent the real world, and coordinates of the robotic arm and the depth sensor in that system are determined by measurements. A linear function (e.g., commonly represented as a transformation matrix) can then transform coordinates of objects as seen by the depth sensor to coordinates of the world (or coordinates of the robotic arm). Such a typical model can have a number of potential error sources. According to an example, errors can result from measuring exact placement of the depth sensor and the robotic arm in a common coordinate frame, which can be difficult at best. Further, the overall system may be prone to falling out of calibration due to mechanical movement of parts within the system. According to another example, while coordinate origins may be determined automatically, linear mapping between reference frames of the robotic arm, the depth sensor, and the world oftentimes can be incorrect. For instance, a depth sensor may have a bias that varies non-linearly with distance or across sensor areas. According to yet another example, a robotic arm can have less than ideal ability to achieve exact placement of an end effector of such robotic arm at a desired Cartesian coordinate. Further, depth sensors or robotic arms, even those from the same vendor, may have slightly different biases, which can further complicate typical approaches for registration.